Pair-wise Separable Quadratic Programming for Constrained Time-varying Regression Estimation

Estimation of time-varying regression model constrained at each time moment by linear inequalities is a natural statistical formulation of a wide class of nonstationary signal processing problems. The presence of linear constraints turns the originally quadratic three-diagonal problem of minimizing the residual squares sum, which is solvable by the linear Kalman-Bucy filtration-smoothing procedure, into that of quadratic programming, which inevitably leads to the necessity of applying much more complicated nonlinear signal processing techniques. However, the threediagonal kind of the quadratic objective function, on one hand, and the specificity of inequality constraints imposed individually upon each vector variable in the sequence of unknown regression coefficients, on the other, essentially simplify the resulting quadratic programming problem in comparison with its standard form. We call problems of such a kind pair-wise separable quadratic programming problem. Two algorithms of nonstationary regression estimation considered in this paper are built as those of pairwise separable quadratic programming and have linear computational complexity in contrast to polynomial complexity of the quadratic programming problem of general kind. The asymptotically strict iterative algorithm is based on the traditional steepest descent method of quadratic programming, whereas the fast approximate algorithm consists in a single run of a special version of the Kalman-Bucy filter-smoother.